Image feature extraction method and system

ABSTRACT

Disclosed is an image feature extraction method including a step of defining a combination of at least two kinds of solid angles along at least two directions, of an input spherical image; a step of determining respective values of the combination of the at least two kinds of solid angles, so that surface areas of spherical crowns of spherical segments, which are obtained by dividing the spherical image by the respective values of the combination of the at least two kinds solid angles, have a same value; and a step of generating, by utilizing the respective values of the combination of the at least two kinds of solid angles, an image feature template so as to conduct image feature extraction.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to the field of image processing, andparticularly relates to an image feature extraction method and an imageextraction system.

2. Description of the Related Art

A panoramic image captured by a panoramic camera, for example, anequirectangular image is usually distorted, especially at the positionsnear two poles, as shown in FIG. 1 which illustrates an example of thepanoramic image captured by the panoramic camera. It can be seen fromFIG. 1 that the objects in the panoramic image are distorted, especiallythe objects at the top and bottom portions; for example, the human faceat the bottom portion is distorted. As a result, in this kind ofdistorted panoramic image, it is difficult to carry out image featureextraction and object detection, especially with respect to thedistorted objects at the positions near two poles.

Generally, in the conventional techniques, there are two types ofmethods for carrying out object detection in the panoramic image. Thefirst type of method is conducting projection transformation to correctthe panoramic image so as to remove distortions thereof, and thenconducting image feature extraction and object detection with respect tothe corrected panoramic image. However, the problem of this type ofmethod is that there doesn't exist a perfect projection transformationapproach by which not only shapes but also straight lines may be kept.Of course, it is also possible to correct the panoramic image byrespectively conducting projection transformation at different positionsthereof, but this kind of approach is inefficient. The second type ofmethod is collecting samples at different positions so as to carry outtraining. For example, it is possible to collect samples at thepositions near two poles of the panoramic image so as to carry out thetraining of image feature extraction, so that it is possible to conductobject detection on the basis of this kind of training. However, theproblem of this type of method is that the process of collecting samplesis too complicated and very difficult, especially with respect to theobjects having various shapes.

Therefore, in order to be able to directly and efficiently conduct imagefeature extraction and object detection with respect to the panoramicimage, it is desirable to propose a new method in which it is notnecessary to conduct projection transformation or collect samples atdifferent positions thereof.

SUMMARY OF THE INVENTION

According to a first aspect of the present invention, an image featureextraction method is provided which includes:

a definition step of defining a combination of at least two kinds ofsolid angles along at least two directions, of an input spherical image;

a determination step of determining respective values of the combinationof the at least two kinds of solid angles, so that surface areas ofspherical crowns of spherical segments, which are obtained by dividingthe spherical image by the respective values of the combination of theat least two kinds solid angles, have a same value; and

a generation step of generating, by utilizing the respective values ofthe combination of the at least two kinds of solid angles, an imagefeature template so as to conduct image feature extraction.

According to a second aspect of the present invention, an image featuredextraction system is provided which includes:

a definition device configured to define a combination of at least twokinds of solid angles along at least two directions, of an inputspherical image;

a determination device configured to determine respective values of thecombination of the at least two kinds of solid angles, so that surfaceareas of spherical crowns of spherical segments, which are obtained bydividing the spherical image by the respective values of the combinationof the at least two kinds solid angles, have a same value; and

a generation device configured to generate, by utilizing the respectivevalues of the combination of the at least two kinds of solid angles, animage feature template so as to conduct image feature extraction.

As a result, by utilizing the image featured extraction method andsystem, it is possible to not only introduce the concept of a solidangle for measuring a spherical surface but also propose a surfaceintegral image for conducting image feature calculation and extraction.Since this kind of method is more efficient, and is more suitable to acase of a spherical surface, it is possible to obtain a better imagefeature extraction result and a better object detection result.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an example of a panoramic image captured by apanoramic camera;

FIG. 2 is a block diagram of a hardware configuration which may utilizevarious embodiments of the present invention;

FIG. 3 is a flowchart of an image feature extraction method according toan embodiment of the present invention;

FIG. 4 is a block diagram of an image feature extraction systemaccording to an embodiment of the present invention;

FIG. 5 is a flowchart of an image feature extraction method according toanother embodiment of the present invention;

FIG. 6A is a flowchart of a process of converting a panoramic image intoa spherical image (i.e., STEP S11) in the method shown in FIG. 5;

FIG. 6B illustrates a detailed process of converting a panoramic imageinto a spherical image;

FIG. 7 is a flowchart of a process of conducting image featureextraction with respect to the spherical image (i.e., STEP S12) in themethod shown in FIG. 5;

FIG. 8A is a flowchart of a process of defining a solid angle pair(i.e., STEP S122) in the process shown in FIG. 7;

FIG. 8B includes images (a) and (b) for describing a detailed process ofdefining a solid angle pair;

FIGS. 8C and 8D illustrate uniqueness and monotonicity of a solid anglepair, respectively;

FIG. 9A is a flowchart of a process of determining the respective valuesof the solid angle pair (i.e., STEP S123) in the process shown in FIG.7;

FIGS. 9B and 9C illustrate a detailed process of determining therespective values of a solid angle pair;

FIG. 9D includes images (a) and (b) which are a longitude andlatitude-based image of a panoramic image and an image obtained afterconducting division by using the respective angles of a solid angle pairwith respect to the longitude and latitude-based image, respectively;

FIG. 10A is a flowchart of a process of conducting image featureextraction on the basis of a surface integral image (i.e., STEP S124) inthe process shown in FIG. 7;

FIGS. 10B to 10E illustrate a detailed process of conducting imagefeature extraction on the basis of a surface integral image;

FIG. 11A is a flowchart of a process of conducting object detection(i.e., STEP S13) in the method shown in FIG. 5; and

FIG. 11B illustrates an example of conducting cascade classifier-basedobject detection.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

In order to let those people skilled in the art better understand thepresent invention, hereinafter the present invention will be concretelydescribed on the basis of the drawings and various embodiments.

FIG. 2 is a block diagram of a hardware configuration 100 which mayutilize various embodiments of the present invention.

For example, it is possible to use a panoramic camera having a CCD(Charge Coupled Device) pickup head 101 to capture a panoramic image,and then to output the captured panoramic image into a digital signalprocessor 103 via a decoder 102. After that, the digital signalprocessor 103 reads necessary data from a storage 104 to conduct imageprocessing, so as to obtain an image feature template, the results ofimage feature extraction, object detection, and object recognition onthe basis of the image feature template, etc. And then, the obtainedresults, etc., may be stored in the storage 104 for storing, may beoutput into a display 105 for displaying, and/or may be sent to acontroller 106 for conducting further processing such as vehicle drivecontrol.

Of course, it is also possible to directly input a spherical image,which is converted from a panoramic image or is captured in a specialway, into the digital signal processor 103 via the decoder 102. In thiscase, the CCD pickup head 101 may be omitted. In addition, the imagefeature extraction method and system according to various embodiments ofthe present invention may be applied to the digital signal processor103.

FIG. 3 is a flowchart of an image feature extraction method according toan embodiment of the present invention.

As shown in FIG. 3, the image extraction method includes STEPS S301 toS303.

STEP S301 is a definition step for defining a combination of at leasttwo kinds of solid angles along at least two directions, of an inputspherical image.

STEP S302 is a determination step for determining the respective valuesof the combination of the at least two kinds of solid angles, so thatthe surface areas of the spherical crowns of the spherical segments,which are obtained by dividing the spherical image by the respectivevalues of the combination of the at least two kinds of solid angles, arethe same (i.e., have a same value).

STEP S303 is a generation step for generating, by utilizing therespective values of the combination of the at least two kinds of solidangles, an image feature template so as to conduct image featureextraction.

Hence, in this way, by defining the combination of the at least twokinds of solid angles (hereinafter, also called a “solid anglecombination”) of a spherical image and determining its respectivevalues, it is possible to reasonably divide the spherical image, so thatthe divided spherical image may have less distortion or may not evenhave distortion. In addition, it is also possible to utilize this kindof solid angle combination as input, so as to generate a proper imagefeature template for carrying out image feature extraction and objectdetection.

Moreover, the image extraction method shown in FIG. 3 may furtherinclude a step of converting a panoramic image so as to obtain thespherical image. That is to say, instead of the spherical image, it isalso possible to input a panoramic image captured by a panoramic camera,and then to convert, by conducting image processing, the panoramic imageinto the spherical image.

In an example, the panoramic image may be an equirectangular image, andthe equirectangular image may be converted into the spherical image byutilizing the following equation (1).

$\begin{matrix}{{r:{\left\lbrack {{- \pi},\pi} \right\rbrack \times \left\lbrack {{- \frac{\pi}{2}},\frac{\pi}{2}} \right\rbrack}}->\left. {R^{2}\left( {\lambda,\phi} \right)}\mapsto\left( {{{\cos(\lambda)}{\cos(\phi)}},{{\sin(\lambda)}{\cos(\phi)}},{\sin(\phi)}} \right) \right.} & (1)\end{matrix}$

Here λ stands for a degree of longitude of the equirectangular image; φstands for a degree of latitude of the equirectangular image; r standsfor mapping from the equirectangular image to the spherical image; and Rstands for the radius of the spherical image.

Of course, the equation (1) is just an example of converting thepanoramic image (i.e., the equirectangular image) into the sphericalimage. In other words, if the panoramic image is another type one, thenit is also possible to adopt a proper known equation to conductconversion from this panoramic image to the spherical image.

Furthermore, the concept of a solid angle is well-known to those peopleskilled in the art. For more information about that, it is also possibleto see https://en.wikipedia.org/wiki/Solid_angle.

In an example, the at least two directions of the solid anglecombination includes any two of the directions along X, Y, and Z axes ofthe spherical image. For instance, it is possible to define two kinds ofsolid angles, namely, a kind of solid angle is along the X axis, andanother kind of solid angle is along the Z axis. In this way, bydefining different kinds of solid angles along different directions, itis possible to simultaneously describe the spherical image along thedifferent directions so as to uniquely utilize the different kinds ofsolid angles to express the spherical image, thereby being capable oflaying a foundation for generating the image feature template andconducting the image feature extraction and object detection on thebasis of the generated image feature template.

In an example, STEP S301 of FIG. 3 may include a step of defining twokinds of solid angles (hereinafter, also called a “solid angle pair”) byutilizing the following equations (2) and (3).

$\begin{matrix}{\theta_{x} = {{2\;{\pi\left( {1 - {\cos\left( {{arc}\;{\cos\left( {\cos\;{{\phi cos}\left( {\frac{\pi}{2} - \lambda} \right)}} \right)}} \right)}} \right)}} = {{2\;{\pi\left( {1 - {\cos\;{\phi sin}\;\lambda}} \right)}} \in \left\lbrack {0,{4\;\pi}} \right\rbrack}}} & (2) \\{\theta_{z} = {{2\;{\pi\left( {1 - {\cos\left( {\frac{\pi}{2} - \phi} \right)}} \right)}} = {{2\;{\pi\left( {1 - {\sin\;\phi}} \right)}} \in \left\lbrack {0,{4\;\pi}} \right\rbrack}}} & (3)\end{matrix}$

Here θ_(x) refers to a kind of solid angle along the X axis; θ_(z)refers to another kind of solid angle along the Z axis; R refers to theradius of the spherical image; λ refers to a degree of longitude of thepanoramic image; and φ refers to a degree of latitude of the panoramicimage.

Of course, this kind of definition is achieved by utilizing the degreesof longitude and latitude of the panoramic image, and is just anexample. That is to say, it is also possible to utilize any other properway to define the solid angle pair.

In addition, the solid angle pair defined in the way above may haveuniqueness and monotonicity. The uniqueness means that regarding eachpoint (e.g., a pixel-based point) on the spherical image, itscorresponding solid angle pair differs from any other one on thespherical image. The monotonicity means that on the panoramic image, thegreater a degree of latitude (when the X-Y plane serves as an equator orthe Y-Z plane serves as an equator) of a point is, the less the value ofa kind of solid angle corresponding to the point is. Of course, theuniqueness and monotonicity are not absolute; for example, it is alsopossible to divide a spherical surface into two semi-spherical surfaces,i.e., front and rear semi-spherical surfaces or upper and lowersemi-spherical surfaces, and to let the uniqueness and monotonicity bemet in each semi-spherical surface.

In an example, STEP S302 of FIG. 3 may include a step of determining therespective values of each kind of solid angle in the solid anglecombination on a semi-spherical surface by utilizing the followingequation (4).

$\begin{matrix}{{\theta(i)} = {\frac{2\; i^{2}}{N^{2}}\pi}} & (4)\end{matrix}$

Here N denotes the number of shapes (spherical segments) obtained bydividing the semi-spherical surface by the respective values of thecorresponding kind of solid angle; θ (i) denotes an i-th value of thecorresponding kind of solid angle; and i denotes a positive integer lessthan or equal to N.

After that, it is possible to symmetrically expand the respective valuesof each kind of solid angle in the solid angle combination on thesemi-spherical surface to the whole spherical surface.

The respective values of each kind of solid angle determined in the wayabove may result in that the surface areas of the spherical crowns ofthe spherical segments, which are obtained by dividing the sphericalimage by the respective values of the solid angle combination, are thesame or evenly distributed. As a result, it is possible to ensure thatthe image of each of the grids (spherical segments) obtained by dividingthe spherical image in the way above has less distortion, or does noteven have distortion, so that it is possible to avoid the distortionproblem occurring in the panoramic image, thereby being able to moreefficiently generate the image feature template, and to more accuratelyconduct the image feature extraction and object detection on the basisof the generated image feature template.

In an example, the image feature template may include a Haar imagefeature template able to be calculated by utilizing an integral image oranother image feature template able to be calculated by utilizing theintegral image. Here it should be noted that utilizing an integral imageto obtain the image feature template is just a simple way of generatingthe image feature template; that is to say, the present invention is notlimited to this. Actually, on the basis of the defined solid anglecombination, it is also possible to adopt another image featuretemplate, for example, a HOG (Histogram of Oriented Gradient) imagefeature template and a SIFT (Scale Invariant Feature Transform) imagefeature template.

In an example, STEP S303 of FIG. 3 may include a step of utilizing thefollowing equation (5) to generate an integral image serving as theimage feature template.

$\begin{matrix}{{{ii}\left( {\theta_{x},\theta_{z}} \right)} = {\sum\limits_{{\theta_{x}^{\prime} \leq \theta_{x}},{\theta_{z}^{\prime} \leq \theta_{z}}}\;{i\left( {\theta_{x}^{\prime},\theta_{z}^{\prime}} \right)}}} & (5)\end{matrix}$

Here i(θ′_(x), θ′_(z)) refers to the gray level of the spherical segmentof the spherical image, defined by θ′_(x) and θ′_(z); ii(θ_(x), θ_(z))refers to the integral image (i.e., a surface integral image); andθ′_(x) and θ′_(z) satisfy the following equation (6).

$\begin{matrix}\left\{ \begin{matrix}{{\theta_{x}^{\prime}(k)} = {\frac{2\; k^{2}}{N^{2}}\pi}} & \left( {{k = 1},2,\ldots\mspace{14mu},N} \right) \\{{\theta_{z}^{\prime}(k)} = {\frac{2\; k^{2}}{N^{2}}\pi}} & \left( {{k = 1},2,\ldots\mspace{14mu},N} \right)\end{matrix} \right. & (6)\end{matrix}$

Up to here, an example of generating the Haar image feature template byutilizing the respective values of the solid angle combination as inputhas been described; however, the present invention is not limited tothis.

In an example, the method shown in FIG. 3 may further include a step ofcarrying out object detection on the basis of an extracted imagefeature. Of course, the object detection is just a kind of applicationconducted by utilizing the extracted image feature; in other words, thepresent invention is not limited to this. Actually, after extracting theimage feature, it is also possible to use the extracted image feature tocarry out object recognition, object tracking, etc.

Hence, by defining the solid value combination of the spherical imageand determining its respective values, it is possible to carry outreasonable division with respect to the spherical image, so that, in thedivided spherical image, there does not exist distortions. In this way,it is possible to avoid the distortion problem occurring in thepanoramic image. In addition, by utilizing this kind of solid anglecombination as input so as to generate a proper image feature templatefor conducting image feature extraction, it is possible to efficiently(i.e., with a small calculation amount and less time) generate the imagefeature template and to more accurately conduct the image featureextraction, so that it is possible to use, for example, the trainingsamples without distortions to carry out a more reliable objectdetection, etc.

FIG. 4 is a block diagram of an image feature extraction system 400according to an embodiment of the present invention.

As shown in FIG. 4, the system 400 includes a definition device 401, adetermination device 402, and a generation device 403.

The definition device 401 is configured to define a combination of atleast two kinds of solid angles along at least two directions, of aninput spherical image.

The determination device 402 is configured to determine the respectivevalues of the combination of the at least two kinds of solid values sothat the surface areas of the spherical crowns of the spherical segmentsobtained by dividing the spherical image by the respective values of thecombination of the at least two kinds of solid values are the same.

The generation device 403 is configured to utilize the respective valuesof the combination of the at least two kinds of solid angles as input soas to generate an image feature template for carrying out image featureextraction.

Hence, by defining the solid angle combination of the spherical imageand determining its respective values, it is possible to reasonablydivide the spherical image so as to reduce or completely remove thedistortions of the divided spherical image. In addition, by utilizingthis kind of solid angle combination as input so as to establish aproper image feature template, it is also possible to achieve functionssuch as image feature extraction and further object detection.

In addition, the system 400 may include a conversion device configuredto conduct conversion with respect to a panoramic image so as to obtainthe spherical image. In other words, instead of the spherical image, itis also possible to directly input a panoramic image captured by apanoramic camera. After that, by utilizing image processing so as toconvert the panoramic image into the spherical image, it is possible tocarry out the follow-on processes.

In an example, the panoramic image may be an equirectangular image whichmay be converted into the spherical image by utilizing the followingequation (1).

$\begin{matrix}{{r:{\left\lbrack {{- \pi},\pi} \right\rbrack \times \left\lbrack {{- \frac{\pi}{2}},\frac{\pi}{2}} \right\rbrack}}->\left. {R^{2}\left( {\lambda,\phi} \right)}\mapsto\left( {{{\cos(\lambda)}{\cos(\phi)}},{{\sin(\lambda)}{\cos(\phi)}},{\sin(\phi)}} \right) \right.} & (1)\end{matrix}$

Here λ stands for a degree of longitude of the panoramic image; φ standsfor a degree of latitude of the panoramic image; r stands for mappingfrom the panoramic image to the spherical image; and R stands for theradius of the spherical image.

Of course, the equation (1) is just an example of converting thepanoramic image (i.e., the equirectangular image) into the sphericalimage. In other words, if the panoramic image is another type one, thenit is also possible to adopt a proper known equation to conductconversion from this panoramic image to the spherical image.

Furthermore, the concept of a solid angle is well-known to those peopleskilled in the art. For more information about that, it is also possibleto see https://en.wikipedia.org/wiki/Solid_angle.

In an example, the at least two directions of the solid anglecombination include any two of the directions along X, Y, and Z axes ofthe spherical image. For instance, it is possible to define two kinds ofsolid angles, namely, a kind of solid angle is along the X axis, andanother kind of solid angle is along the Z axis. In this way, bydefining different kinds of solid angles along different directions, itis possible to simultaneously describe the spherical image along thedifferent directions so as to uniquely utilize the different kinds ofsolid angles to express the spherical image, thereby being capable oflaying a foundation for generating the image feature template andconducting the feature extraction and object detection on the basis ofthe generated image feature template.

In an example, the definition device 401 of the system 400 may beconfigured to define two kinds of solid angles by utilizing thefollowing equations (2) and (3).

$\begin{matrix}{\theta_{x} = {{2\;{\pi\left( {1 - {\cos\left( {{arc}\;{\cos\left( {\cos\;{{\phi cos}\left( {\frac{\pi}{2} - \lambda} \right)}} \right)}} \right)}} \right)}} = {{2\;{\pi\left( {1 - {\cos\;{\phi sin}\;\lambda}} \right)}} \in \left\lbrack {0,{4\;\pi}} \right\rbrack}}} & (2) \\{\theta_{z} = {{2\;{\pi\left( {1 - {\cos\left( {\frac{\pi}{2} - \phi} \right)}} \right)}} = {{2\;{\pi\left( {1 - {\sin\;\phi}} \right)}} \in \left\lbrack {0,{4\;\pi}} \right\rbrack}}} & (3)\end{matrix}$

Here θ_(x) refers to a kind of solid angle along X axis; θ_(z) refers toanother kind of solid angle along Z axis; R refers to the radius of thespherical image; λ refers to a degree of longitude of the panoramicimage; and φ refers to a degree of latitude of the panoramic image.

Of course, this kind of definition is achieved by utilizing the degreesof longitude and latitude of the panoramic image, and is just anexample. That is to way, it is also possible to utilize any other properway to define the solid angle pair.

In addition, the solid angle pair defined in the way above may haveuniqueness and monotonicity. The uniqueness means that regarding eachpoint (e.g., a pixel-based point) on the spherical image, itscorresponding solid angel pair differs from any other one on thespherical image. The monotonicity means that on the panoramic image, thegreater a degree of latitude (when the X-Y plane serves as an equator orthe Y-Z plane serves as an equator) of a point is, the less the value ofa kind of solid angle corresponding to the point is. Of course, theuniqueness and monotonicity are not absolute; for example, it is alsopossible to divide a spherical surface into two semi-spherical surfaces,i.e., front and rear semi-spherical surfaces or upper and lowersemi-spherical surfaces, and to let the uniqueness and monotonicity bemet in each semi-spherical surface.

In an example, the determination device 402 of the system 400 may beconfigured to determine the respective values of each kind of solidangle in the solid angle combination on a semi-spherical surface byutilizing the following equation (4).

$\begin{matrix}{{\theta(i)} = {\frac{2\; i^{2}}{N^{2}}\pi}} & (4)\end{matrix}$

Here N denotes the number of shapes (spherical segments) obtained bydividing the semi-spherical surface by the respective values of thecorresponding kind of solid angle; θ(i) denotes an i-th value of thecorresponding kind of solid angle; and i denotes a positive integer lessthan or equal to N.

After that, it is possible to symmetrically expand the respective valuesof each kind of solid angle in the solid angle combination on thesemi-spherical surface to the whole spherical surface.

The respective values of each kind of solid angle determined in the wayabove may result in that the surface areas of the spherical crowns ofthe spherical segments, which are obtained by dividing the sphericalimage by the respective values of the solid angle combination, are thesame or evenly distributed. As a result, it is possible to ensure thatthe image of each of the grids (spherical segments) obtained by dividingthe spherical image in the way above has less distortion, or does noteven have distortion, so that it is possible to avoid the distortionproblem occurring in the panoramic image, thereby being able to moreefficiently generate the image feature template, and to more accuratelyconduct the image feature extraction and object detection on the basisof the generated image feature template.

In an example, the image feature template may include a Haar imagefeature template being able to be calculated by utilizing an integralimage or another image feature template being able to be calculated byutilizing the integral image. Here it should be noted that utilizing anintegral image to obtain the image feature template is just a simple wayof generating the image feature template; that is to say, the presentinvention is not limited to this. Actually, on the basis of the definedsolid angle combination, it is also possible to adopt another imagefeature template, for example, a HOG (Histogram of Oriented Gradient)image feature template and a SIFT (Scale Invariant Feature Transform)image feature template.

In an example, the generation device 403 of the system 400 may beconfigured to utilize the following equation (5) to generate an integralimage serving as the image feature template.

$\begin{matrix}{{{ii}\left( {\theta_{x},\theta_{z}} \right)} = {\sum\limits_{{\theta_{x}^{\prime} \leq \theta_{x}},{\theta_{z}^{\prime} \leq \theta_{z}}}\;{i\left( {\theta_{x}^{\prime},\theta_{z}^{\prime}} \right)}}} & (5)\end{matrix}$

Here i(θ′_(x), θ′_(z)) refers to the gray level of the spherical segmentof the spherical image, defined by θ′_(x) and θ′_(z); ii(θ_(x), θ_(z))refers to the integral image (i.e., a surface integral image); andθ′_(x) and θ′_(z) satisfy the following equation (6).

$\begin{matrix}\left\{ \begin{matrix}{{\theta_{x}^{\prime}(k)} = {\frac{2\; k^{2}}{N^{2}}\pi}} & \left( {{k = 1},2,\ldots\mspace{14mu},N} \right) \\{{\theta_{z}^{\prime}(k)} = {\frac{2\; k^{2}}{N^{2}}\pi}} & \left( {{k = 1},2,\ldots\mspace{14mu},N} \right)\end{matrix} \right. & (6)\end{matrix}$

Up to here, an example of generating the Haar image feature template byutilizing the respective values of the solid angle combination as inputhas been described; however, the present invention is not limited tothis.

In an example, the system 400 shown in FIG. 4 may further include adetection device configured to carry out object detection on the basisof an extracted image feature. Of course, the object detection is just akind of application conducted by utilizing the extracted image feature;in other words, the present invention is not limited to this. Actually,after extracting the image feature, it is also possible to use theextracted image feature to carry out object recognition, objecttracking, etc.

Hence, by defining the solid value combination of the spherical imageand determining it respective values, it is possible to carry outreasonable division with respect to the spherical image, so that, in thedivided spherical image, there does not exist distortions. In this way,it is possible to avoid the distortion problem occurring in thepanoramic image. In addition, by utilizing this kind of solid anglecombination as input so as to generate a proper image feature templatefor conducting image feature extraction, it is possible to efficiently(i.e., with a small calculation amount and less time) generate the imagefeature template and to more accurately conduct the image featureextraction, so that it is possible to use, for example, the trainingsamples without distortions to carry out a more reliable objectdetection, etc.

FIG. 5 is a flowchart of an image feature extraction method according toanother embodiment of the present invention.

As shown in FIG. 5, the method includes at least STEPS S11 to S13. InSTEP S11, a panoramic image is converted into a spherical image. In STEPS12, image feature extraction is conducted with respect to the sphericalimage. In STEP S13, object detection is carried out.

Here it should be noted that what the method shown in FIG. 5 includes isjust an example; that is to say, the present invention is not limited tothis. For example, STEP S11 is for obtaining the spherical image just ina case where the panoramic image is input in STEP S10. In addition, STEPS13 is just for conducting the object detection if it needs to be done.In this case, the method shown in FIG. 5 may further include STEP S14.

FIG. 6A is a flowchart of a process of converting a panoramic image intoa spherical image (i.e., STEP S11) in the method shown in FIG. 5.

As shown in FIG. 6A, in STEP S111, the panoramic image is input. Andthen, in STEP S112, the panoramic image is converted into the sphericalimage (i.e., coordinate conversion from the panoramic image into thespherical image is conducted). After that, the converted spherical imageis output. An example of the coordinate conversion equation as follows.

$\begin{matrix}{{r:{\left\lbrack {{- \pi},\pi} \right\rbrack \times \left\lbrack {{- \frac{\pi}{2}},\frac{\pi}{2}} \right\rbrack}}->\left. {R^{2}\left( {\lambda,\phi} \right)}\mapsto\left( {{{\cos(\lambda)}{\cos(\phi)}},{{\sin(\lambda)}{\cos(\phi)}},{\sin(\phi)}} \right) \right.} & (1)\end{matrix}$

Here λ stands for a degree of longitude of the panoramic image; φ standsfor a degree of latitude of the panoramic image; r stands for mappingfrom the panoramic image to the spherical image; and R stands for theradius of the spherical image. In addition, the panoramic imageprocessed by the coordinate conversion equation is an equirectangularimage, and the coordinate conversion equation is one in a sphericalcoordinate system.

Here it should be noted that if another type of panoramic image isinput, then it is possible to adopt a proper conversion equationcorresponding to this panoramic image. Furthermore, if a spherical imageis input, then it is also possible to omit this step.

FIG. 6B illustrates a detailed process of converting a panoramic imageinto a spherical image.

FIG. 6B includes first and second images from left to right, where thefirst image illustrates a panoramic image which is a plane similar to arectangle having longitude and latitude coordinates (λ, φ). By utilizingthe equation (1) above to convert the panoramic image, it is possible toobtain a spherical image illustrated by the second image, which is aspherical surface expressed by three directions of X, Y, and Z axes. Thelongitude and latitude coordinates (λ, φ) are mapped into correspondingpositions of the spherical image.

FIG. 7 is a flowchart of a process of conducting image featureextraction with respect to the spherical image (i.e., STEP S12) in themethod shown in FIG. 5.

As shown in FIG. 7, STEP S12 of the method shown in FIG. 5 may includeSTEPS S121 to S125. In STEP S121, the converted spherical image isobtained as input. In STEP S122, a solid angle pair is defined. In STEPS123, the respective values of the solid angle pair are determined. InSTEP S124, on the basis of an integral image, the image featureextraction is conducted. In STEP S125, the extracted image feature isoutput.

Here it should be noted that defining the solid angle pair (i.e., twokinds of solid angles) is just an example. Actually it is also to definemore kinds of solid angles so as to carry out more accurate sphericalsurface division and image feature extraction.

FIG. 8A is a flowchart of a process of defining the solid angle pair(i.e., STEP S122) in the process shown in FIG. 7.

As shown in FIG. 8A, the process includes STEPS S1221 to S1224. In STEPS1221, the converted spherical image is input. In STEP S1222, a kind ofsolid angle related to the X axis is defined. In STEP S1223, a kind ofsolid angle related to the Z axis is defined. In STEP S1224, acombination of the defined two kinds of solid angles (i.e., the solidangle pair) is out.

FIG. 8B includes images (a) and (b) for describing a detailed process ofdefining a solid angle pair.

As shown in FIG. 8B, the image (a) illustrates an example of a kind ofsolid angle related to the X axis, which is restricted by a circle EFthat is the surface of a spherical crown AEF. The image (b) illustratesan example of another kind of solid angle related to the Z axis, whichis restricted by a circle CD that is the surface of a spherical crownBCD.

The circle EF is perpendicular to the X axis, and its correspondingsolid angle θ_(x) may be defined by the following equation.

$\begin{matrix}{\theta_{x} = {\frac{S_{x}}{R^{2}} = {\frac{S_{AEF}}{R^{2}} = {{2\;{\pi\left( {1 - {\cos\left( {{arc}\;{\cos\left( {\cos\;{{\phi cos}\left( {\frac{\pi}{2} - \lambda} \right)}} \right)}} \right)}} \right)}} = {{2\;{\pi\left( {1 - {\cos\;{\phi sin}\;\lambda}} \right)}} \in \left\lbrack {0,{4\;\pi}} \right\rbrack}}}}} & (2)\end{matrix}$

The circle CD is perpendicular to the Z axis, and its correspondingsolid angle θ_(z) may be defined by the following equation.

$\begin{matrix}{\theta_{z} = {\frac{S_{z}}{R^{2}} = {\frac{S_{BCD}}{R^{2}} = {{2{\pi\left( {1 - {\cos\left( {\frac{\pi}{2} - \phi} \right)}} \right)}} = {{2{\pi\left( {1 - {\sin\;\phi}} \right)}} \in \left\lbrack {0,{4\pi}} \right\rbrack}}}}} & (3)\end{matrix}$

In the above two equations, λ refers to a degree of longitude; φ refersto a degree of latitude; S_(AEF) refers to the surface area of thespherical crown AEF, as shown in the image (a) of FIG. 8B; S_(BCD)refers to the surface area of the spherical crown BCD, as shown in theimage (b) of FIG. 8B; θ_(x) refers to the kind of solid angle related tothe X axis; and θ_(z) refers to the other kind of solid angle related tothe Z axis. According the above two equations, it is apparent that therange of each of θ_(x) and θ_(z) is within [0,4π].

The solid angle combination (also called a “solid angle pair”) definedin the way above may have uniqueness and monotonicity.

FIGS. 8C and 8D illustrate the uniqueness and monotonicity of the solidangle pair, respectively.

According to the image (a) of FIG. 8A, it can be seen that regarding thevalues of θ_(x) corresponding to the all points on the circle EF, theyare the same; however, regarding the values of θ_(z) corresponding tothe all points on the same circle, except some solid angle pairs asshown in FIG. 8C, they are different. Here, according to FIG. 8C, it canbe seen that, for example, the solid angle pairs (θ_(x), θ_(z))corresponding to the points P and Q are the same. In this case, it ispossible to divide the spherical surface into front and rearsemi-spherical surfaces. Thus, each of the semi-spherical surfaces mayhave the uniqueness.

In addition, according to FIG. 8D, it can be seen that regarding the twopoints r and t, the values of θ_(x) corresponding to them are the same;however, the value of θ_(z) corresponding to the point r is less thanthe value of θ_(z) corresponding to the point t, and the degree oflatitude corresponding to the point r is greater than that correspondingto the point t. That is to say, the greater the degree of latitude is,the less the corresponding value of θ_(z) is.

FIG. 9A is a flowchart of a process of determining the respective valuesof the solid angle pair (i.e., STEP S123) in the process shown in FIG.7.

As shown in FIG. 9A, the process includes STEPS S1231 to S1234. In STEPS1231, the spherical image and the defined two kinds of solid angles areinput. In STEP S1232, the spherical image is divided by utilizing thedefined two kinds of solid angles. In STEP S1233, the value distribution(i.e., the respective values) of each kind of solid angle is determined.In STEP S1234, the respective values of each kind of solid angleobtained by calculation are output.

FIGS. 9B and 9C illustrate a detailed process of determining therespective values of a solid angle pair.

As shown in FIG. 9B, the upper semi-spherical surface of a sphericalimage is divided into three annular sections by utilizing the respectivevalues of a kind of solid angle related to the Z axis; as a result, thespherical image is divided into five (i.e., 3×2−1=5) annular sectionsalong the Z axis in total. In the same way, the right semi-sphericalsurface of the spherical image is divided into three annular sections byutilizing the respective values of a kind of solid angle related to theX axis; as a result, the spherical image is divided into five (i.e.,3×2−1=5) annular sections along the X axis in total.

Furthermore, as shown in FIG. 9B, the circles on the spherical surfaceperpendicular to the X axis are tangent to the circles on the sphericalsurface perpendicular to the Z axis, respectively, so as to generateplural spherical segments. As described above, according to STEP S1233of FIG. 9A, it is possible to determine the value distribution of eachkind of solid angle. As shown in FIG. 9B, it is assumed that the area ofeach spherical segment is S. Since the spherical crown corresponding toθ₃, which is one of respective values of the kind of solid angle θ_(z),is a semi-spherical surface, it is possible to calculate the value of θ₃as follows.

$\begin{matrix}{\theta_{3} = {\frac{S}{R^{2}} = {\frac{2\pi\; R^{2}}{R^{2}} = {2\pi}}}} & (7)\end{matrix}$

Here R stands for the radius of the spherical image.

On the basis of the above spherical surface division, it is possible toobtain the number of pixels corresponding to each value of the kind ofsolid angle θ_(z) (e.g., θ₁, θ₂, and θ₃). After that, the respectivevalues of θ_(z) may be obtained by conducting the following calculation.

$\begin{matrix}\left\{ \begin{matrix}{S_{A} = {{\theta_{1}R^{2}} = {2\; S}}} \\{S_{AB} = {{S_{B} - S_{A}} = {{\left( {\theta_{2} - \theta_{1}} \right)R^{2}} = {6\; S}}}} \\{S_{BC} = {{S_{C} - S_{B}} = {{\left( {\theta_{3} - \theta_{2}} \right)R^{2}} = {10\; S}}}}\end{matrix}\rightarrow\left\{ {\left. \begin{matrix}{\theta_{1} = {\frac{2}{9}\pi}} \\{\theta_{2} = {\frac{8}{9}\pi}} \\{\theta_{3} = {2\pi}}\end{matrix}\rightarrow{\theta(i)} \right. = {{\frac{2\; i^{2}}{9}\pi} = {\frac{2\; i^{2}}{N^{2}}\pi}}} \right. \right. & (8)\end{matrix}$

Here S_(A) denotes the surface area of the spherical crown correspondingto the solid angle value θ₁; S_(B) denotes the surface area of thespherical crown corresponding to the solid angle value θ₂; S_(C) denotesthe surface area of the spherical crown corresponding to the solid anglevalue θ₃; S_(AB) denotes the surface area of the annular sphericalsurface AB; S_(BC) denotes the surface area of the annular sphericalsurface BC; N denotes the number of the circles on a semi-sphericalsurface perpendicular to the Z axis, and the total number of the circleson the spherical surface perpendicular to the Z axis is 2N−1; and, inthis example, N=3.

In addition, the spherical surface may also be divided as shown in FIG.9C. If it is assumed that the area of each spherical segment is S, thenin the same way as above, it is also possible to obtain the values ofθ₆, θ₄, and θ₂; that is, θ₆=2π, θ₄=(8/9)π, and θ₂=(2/9)π. As a result,on the basis of this kind of spherical surface division, it is alsopossible to obtain the number of pixels corresponding to each value ofθ_(z) (e.g., θ₁, θ₂, θ₄, and θ₅). After that, the respective values ofthe kind of solid angle θ_(z) may be acquired by carrying out thefollowing calculation.

$\begin{matrix}{{{\theta_{6} = {2\pi}};{\theta_{4} = {\frac{8}{9}\pi}};{\theta_{2} = {\frac{2}{9}\pi}};{N = 6}}\left\{ \begin{matrix}{S_{A} = {{\theta_{1}R^{2}} = {2\; S}}} \\{S_{AB} = {{S_{B} - S_{A}} = {{\left( {\theta_{2} - \theta_{1}} \right)R^{2}} = {6\; S}}}} \\{S_{BC} = {{S_{C} - S_{B}} = {{\left( {\theta_{3} - \theta_{2}} \right)R^{2}} = {10\; S}}}} \\{S_{CD} = {{S_{D} - S_{C}} = {{\left( {\theta_{4} - {\theta 3}} \right)R^{2}} = {14\; S}}}} \\{S_{DE} = {{S_{E} - S_{D}} = {{\left( {\theta_{5} - \theta_{4}} \right)R^{2}} = {18\; S}}}} \\{S_{EF} = {{S_{F} - S_{E}} = {{\left( {\theta_{6} - \theta_{5}} \right)R^{2}} = {22\; S}}}}\end{matrix}\rightarrow\left\{ {\left. \begin{matrix}{\theta_{1} = {\frac{1}{18}\pi}} \\{\theta_{2} = {\frac{2}{9}\pi}} \\{\theta_{3} = {\frac{1}{2}\pi}} \\{\theta_{4} = {\frac{8}{9}\pi}} \\{\theta_{5} = {\frac{25}{18}\pi}} \\{\theta_{6} = {2\pi}}\end{matrix}\rightarrow{\theta(i)} \right. = {{\frac{i^{2}}{18}\pi} = {\frac{2\; i^{2}}{N^{2}}\pi}}} \right. \right.} & (9)\end{matrix}$

Here S_(A) stands for the surface area of the spherical crowncorresponding to the solid angle value θ₁; S_(B) stands for the surfacearea of the spherical crown corresponding to the solid angle value θ₂;S_(C) stands for the surface area of the spherical crown correspondingto the solid angle value θ₃; S_(D) stands for the surface area of thespherical crown corresponding to the solid angle value θ₄; S_(E) standsfor the surface area of the spherical crown corresponding to the solidangle value θ₅; S_(E) stands for the surface area of the spherical crowncorresponding to the solid angle value θ₆; S_(AB) stands for the surfacearea of the annular spherical surface AB; S_(BC) stands for the surfacearea of the annular spherical surface BC; S_(CD) stands for the surfacearea of the annular spherical surface CD; S_(DE) stands for the surfacearea of the annular spherical surface DE; and S_(EE) stands for thesurface area of the annular spherical surface EF. In addition, N standsfor the number of the circles on one of the two semi-spherical surfaces,perpendicular to the Z axis; as a result, the total number of thecircles on the whole spherical surface, perpendicular to the Z axis is2N−1. In this example, N=6.

According to the equations (8) and (9), it can be seen that the solidangle value calculation equations are the same.

Furthermore, it is possible to divide the spherical surface and tocalculate the corresponding solid angle values. Here it should be notedthat the calculated solid angle values also satisfy the followingequation.

$\begin{matrix}{{\theta(i)} = {\frac{2\; i^{2}}{N^{2}}\pi}} & (4)\end{matrix}$

Here N refers to the number of shapes (spherical segments) obtained bydividing a semi-spherical surface by the respective values of each solidangle value; θ(i) refers to the i-th value of the corresponding kind ofsolid angle; and i is a positive integer which is less than or equal toN.

Moreover, regarding a case of a multiple of N (e.g., 2N or 4N), bycarrying out the calculation in the same way as above, it is alsopossible to obtain the equation (4), i.e.,

${\theta(i)} = {\frac{2\; i^{2}}{N^{2}}{\pi.}}$

After that, by symmetrically expanding the respective values θ(i) on thesemi-spherical surface to the whole spherical surface, it is possible toacquire the solid angle value distribution on the whole sphericalsurface.

Here it should be noted that since it is assumed that the areas of therespective segments are the same, and because the respective solid anglevalues are calculated on the basis of the premise, the sizes of therespective spherical segments on the spherical surface divided by thecalculated solid angle values should be the same. In addition, each ofthe shapes of most spherical segments approaches a grid. As a result, itis possible to reach a conclusion that the spherical segments obtainedby dividing the whole spherical surface by the calculated values of thedefined solid angle pair are uniform. This is the advantage of thedefined solid angel pair.

FIG. 9D includes images (a) and (b) which are a longitude andlatitude-based image of a panoramic image and an image obtained afterconducting division by using the respective angles of a solid angle pairwith respect to the longitude and latitude-based image, respectively.

As shown in FIG. 9D, in the image (a), the grids are not uniform.However, in the image (b), the grids are uniform. In addition, it can beseen from the images (a) and (b) that the human face in the image (a) isdeformed, but the same human face in the image (b) is not deformed.

FIG. 10A is a flowchart of a process of conducting image featureextraction on the basis of a surface integral image (i.e., STEP S124) inthe process shown in FIG. 7.

FIGS. 10B to 10E illustrate a detailed process of conducting imagefeature extraction on the basis of a surface integral image.

As shown in FIG. 10A, STEP S124 of FIG. 7 includes STEPS S1241 to S1245.In STEP S1241, the spherical image and the calculated respective valuesof the solid angle pair are obtained so as to serve as input. In STEPS1242, an image feature template is defined. In STEP S1243, the surfaceintegral image is calculated. In STEP S1244, image feature values arecalculated. In STEP S1245, the calculated image feature values areoutput.

Particularly, in STEP S1242, the image feature template may be one ofHaar feature templates shown in FIG. 10B or any other proper imagefeature template. The present invention is not limited to this.

As shown in FIG. 10B, it is possible to use a simple rectangularcombination to serve as the image feature template. This kind of featuretemplate may be formed by two or more congruent adjacent rectangles. Inthis kind of image feature template, there are two types of rectangles,i.e., white and black types, which are arranged in an interleaved way.In addition, the image feature value of this kind of image featuretemplate is defined as the result of subtracting the sum of pixels inthe black rectangle from the sum of pixels in the white rectangle.Particularly, in FIG. 10B, there are four simple image feature templatesA, B, C, and D. The image features templates A and B stand for edgefeatures, the image feature template C stands for a linear feature, andthe image feature template D stands for a specific direction feature.

As shown in FIG. 10C, a typical algorithm of calculating an integralimage is given. Here it should be noted that the concept of an integralimage is similar to the concept of integral in calculus; both of themare calculating the area of a rectangle formed by a top-left pointserving as an origin and a bottom-right point. In particular, regardinga point (x,y) in an image, its integral image ii(x,y) may be defined asfollows.

${{ii}\left( {x,y} \right)} = {\sum\limits_{{x^{\prime} \leq x},{y^{\prime} \leq y}}\;{i\left( {x^{\prime},y^{\prime}} \right)}}$

Here i(x′,y′) is called an “original image” at a point (x′,y′), and isthe color value of the point; in a case of a grayscale image, its valueis from 0 to 255.

It is thus apparent that by scanning the image only once, it is possibleto obtain the whole integral image matrix.

After that, as shown in FIG. 10C, it may be known that ii(1)=the sum ofpixels in a region A, ii(2)=the sum of pixels in regions A and B,ii(3)=the sum of pixels in regions A and C, ii(4)=the sum of pixels inregions A, B, C, and D. As a result, it is possible to know that the sumof pixels of the region D=ii(1)+ii(4)−(ii(2)+ii(3)).

Here it should be noted that for more information about the integralimage, it is also possible to seehttps://en.wikipedia.org/wiki/Integral_image.

For example, in a case of the image feature template A shown in FIG.10B, the image feature value of the image feature template A is definedas “the sum of pixels in the white rectangle (region)−the sum of pixelsin the black rectangle (region)”, as described above.

In this case, since the sum of pixels in the white rectangle (region)may be calculated as ii(4)+ii(1)−(ii(2)+ii(3)), and because the sum ofpixels in the black rectangle (region) may be calculated asii(6)+ii(3)−(ii(4)+ii(5)), the image feature value of the image featuretemplate A shown in FIG. 10B may be obtained as(ii(4)−ii(3))−(ii(2)−ii(1))+(ii(4)−ii(3))−(ii(6)−ii(5))=S(E)−S(D), asshown in FIG. 10D. Here S refers to an area.

In the same way, in a case of the spherical image used in theembodiments of the present invention, as shown in FIG. 10E, if thespherical segments, which are obtained by dividing the spherical imageby the respective values of the solid angle combination, are consideredas rectangles, then the corresponding feature value may be obtained asS(E)−S(D)=(ii(5)+ii(2))−(ii(6)+ii(1))−(ii(1)+ii(4))−(ii(2)+ii(3)), asshown in FIG. 10E.

As a result, in STEP S1243 of FIG. 10A, when calculating the sphericalintegral image, it may be defined as follows.

$\begin{matrix}{{{ii}\left( {\theta_{x},\theta_{z}} \right)} = {\sum\limits_{{\theta^{\prime} \leq \theta_{x}},{\theta^{\prime} \leq \theta_{z}}}\;{i\left( {\theta_{x}^{\prime},\theta_{z}^{\prime}} \right)}}} & (5)\end{matrix}$

Here i(θ′_(x), θ′_(z)) stands for a gray level of a spherical segment inthe spherical image, defined by θ′_(x), and θ′_(z); ii(θ_(X), θ_(Z))stands for the spherical integral image; and θ′_(x) and θ′_(z) need tosatisfy the following equation.

$\begin{matrix}\left\{ \begin{matrix}{{\theta_{x}^{\prime}(k)} = {\frac{2\; k^{2}}{N^{2}}\pi}} & \left( {{k = 1},2,\ldots\mspace{14mu},N} \right) \\{{\theta_{z}^{\prime}(k)} = {\frac{2\; k^{2}}{N^{2}}\pi}} & \left( {{k = 1},2,\ldots\mspace{14mu},N} \right)\end{matrix} \right. & (6)\end{matrix}$

After that, as described above, by scanning the whole spherical image,it is possible to acquire the whole integral image matrix of thespherical image used in the embodiments of the present invention, sothat it is possible to utilize, for example, the image feature templateA shown in FIG. 10B to calculate an image feature value, and let thecalculated feature value serve as an image feature.

Here it should be noted that although the image feature template A amongthe Haar feature templates shown in FIG. 10B is taken as an example fordescription, the present invention is not limited to this.

FIG. 11A is a flowchart of a process of conducting object detection(i.e., STEP S13) in the method shown in FIG. 5.

As shown in FIG. 11A, the process may include STEPS S131 to S133. InSTEP S131, the extracted image feature is input. In STEP S132, cascadeclassifier-based object detection is conducted. In STEP S133, the objectdetection result is output.

FIG. 11B illustrates an example of conducting cascade classifier-basedobject detection.

Here, since this kind of cascade classifier-based object detection iswell-known in the art, its related description is omitted.

Therefore, in this way, by defining the solid angle combination of aspherical image and determining its respective values, it is possible toreasonably divide the spherical image, so that the divided sphericalimage may have less distortion or may not even have distortion. Inaddition, it is also possible to utilize this kind of solid anglecombination as input so as to generate a proper image feature templatefor carrying out feature extraction and object detection, so that it ispossible to conduct more efficient template generation, more accuratefeature extraction, and more reliable object detection.

Here it should be noted that the above respective embodiments are justexemplary ones, and the specific structure and operation of each of themmay not be used for limiting the present invention.

Moreover, the embodiments of the present invention may be implemented inany convenient form, for example, using dedicated hardware or a mixtureof dedicated hardware and software. The embodiments of the presentinvention may be implemented as computer software implemented by one ormore networked processing apparatuses. The network may comprise anyconventional terrestrial or wireless communications network, such as theInternet. The processing apparatuses may comprise any suitablyprogrammed apparatuses such as a general-purpose computer, personaldigital assistant, mobile telephone (such as a WAP or 3G-compliantphone) and so on. Since the embodiments of the present invention can beimplemented as software, each and every aspect of the present inventionthus encompasses computer software implementable on a programmabledevice.

The computer software may be provided to the programmable device usingany storage medium for storing processor-readable code such as a floppydisk, a hard disk, a CD ROM, a magnetic tape device or a solid statememory device.

The hardware platform includes any desired hardware resources including,for example, a central processing unit (CPU), a random access memory(RAM), and a hard disk drive (HDD). The CPU may include processors ofany desired type and number. The RAM may include any desired volatile ornonvolatile memory. The HDD may include any desired nonvolatile memorycapable of storing a large amount of data. The hardware resources mayfurther include an input device, an output device, and a network devicein accordance with the type of the apparatus. The HDD may be providedexternal to the apparatus as long as the HDD is accessible from theapparatus. In this case, the CPU, for example, the cache memory of theCPU, and the RAM may operate as a physical memory or a primary memory ofthe apparatus, while the HDD may operate as a secondary memory of theapparatus.

While the present invention is described with reference to the specificembodiments chosen for purpose of illustration, it should be apparentthat the present invention is not limited to these embodiments, butnumerous modifications could be made thereto by those people skilled inthe art without departing from the basic concept and technical scope ofthe present invention.

The present application is based on and claims the benefit of priorityof Chinese Patent Application No. 201410386817.7 filed on Aug. 7, 2014,the entire contents of which are hereby incorporated by reference.

What is claimed is:
 1. An image feature extraction method, comprising: a definition step of defining a combination of at least two kinds of solid angles along at least two directions, of an input spherical image; a determination step of determining respective values of the combination of the at least two kinds of solid angles, so that surface areas of spherical crowns of spherical segments, which are obtained by dividing the spherical image by the respective values of the combination of the at least two kinds of solid angles, having the same value; and a generation step of generating, by utilizing the respective values of the combination of the at least two kinds of solid angles, an image feature template so as to conduct image feature extraction.
 2. The method according to claim 1, further comprising: a conversion step of converting a panoramic image so as to obtain the spherical image.
 3. The method according to claim 2, wherein: the panoramic image is an equirectangular image, and the equirectangular image is converted into the spherical image by utilizing $\begin{matrix} \left. {r{\text{:}\mspace{14mu}\left\lbrack {{- \pi},\pi} \right\rbrack} \times \left\lbrack {{- \frac{\pi}{2}},\frac{\pi}{2}} \right\rbrack}\rightarrow R^{2} \right. \\ \left. \left( {\lambda,\phi} \right)\mapsto\left( {{{\cos(\lambda)}{\cos(\phi)}},{{\sin(\lambda)}{\cos(\phi)}},{\sin(\phi)}} \right) \right. \end{matrix},$ wherein, λ stands for a degree of longitude of the equirectangular image; φ stands for a degree of latitude of the equirectangular image; r stands for mapping from the equirectangular image to the spherical image; and R stands for a radius of the spherical image.
 4. The method according to claim 1, wherein: the at least two directions of the combination of the at least two kinds of solid angles include at least two among directions along X, Y, and Z axes of the spherical image.
 5. The method according to claim 4, wherein, the definition step includes: utilizing ${\theta_{x} = {{2{\pi\left( {1 - {\cos\left( {\arccos\left( {\cos\;\phi\;{\cos\left( {\frac{\pi}{2} - \lambda} \right)}} \right)} \right)}} \right)}} = {{2{\pi\left( {1 - {\cos\;\phi\;\sin\;\lambda}} \right)}} \in \left\lbrack {0,{4\pi}} \right\rbrack}}},$ and $\theta_{z} = {{2{\pi\left( {1 - {\cos\left( {\frac{\pi}{2} - \phi} \right)}} \right)}} = {{2{\pi\left( {1 - {\sin\;\phi}} \right)}} \in \left\lbrack {0,{4\pi}} \right\rbrack}}$ to define two kinds of solid angles, wherein, θ_(x) refers to a kind of solid angle along the X axis; θ_(z) refers to another kind of solid angle along the Z axis; R refers to a radius of the spherical image; λ refers to a degree of longitude of a panoramic image; and φ refers to a degree of latitude of a panoramic image.
 6. The method according to claim 1, wherein, the determination step includes: utilizing ${\theta(i)} = {\frac{2\; i^{2}}{N^{2}}\pi}$ to determine respective values of each kind of solid angle of the combination of the at least two kinds of solid angles on a semi-spherical surface, wherein, N stands for the number of spherical segments obtained by dividing the semi-spherical surface by the respective values of the corresponding kind of solid angle, θ(i) stands for an i-th value of the corresponding kind of solid angle, and i stands for a positive integer less than or equal to N; and symmetrically expanding the respective values of each kind of solid angle of the combination of the at least two kinds of solid angles on a semi-spherical surface to the whole spherical surface.
 7. The method according to claim 1, wherein: the image feature template includes an integral image-based Haar feature template.
 8. The method according to claim 6, wherein, the generation step includes: utilizing ${{ii}\left( {\theta_{x},\theta_{z}} \right)} = {\sum\limits_{{\theta^{\prime} \leq \theta_{x}},{\theta^{\prime} \leq \theta_{z}}}\;{i\left( {\theta_{x}^{\prime},\theta_{z}^{\prime}} \right)}}$ to generate a spherical integral image to serve as the image feature template, wherein, i(θ′_(x), θ′_(z)) stands for a gray level of a spherical segment on the spherical image, defined by θ′_(x) and θ′_(z); ii(θ_(X), θ_(Z)) stands for the spherical integral image; and θ′_(x) and θ′_(z) $\left\{ {\begin{matrix} {{\theta_{x}^{\prime}(k)} = {\frac{2\; k^{2}}{N^{2}}\pi}} & \left( {{k = 1},2,\ldots\mspace{14mu},N} \right) \\ {{\theta_{z}^{\prime}(k)} = {\frac{2\; k^{2}}{N^{2}}\pi}} & \left( {{k = 1},2,\ldots\mspace{14mu},N} \right) \end{matrix}.} \right.$
 9. The method according to claim 1, further comprising: a detection step of conducting object detection based on an extracted image feature.
 10. An image feature extraction system, comprising: a definition device configured to define a combination of at least two kinds of solid angles along at least two directions, of an input spherical image; a determination device configured to determine respective values of the combination of the at least two kinds of solid angles, so that surface areas of spherical crowns of spherical segments, which are obtained by dividing the spherical image by the respective values of the combination of the at least two kinds of solid angles, having the same value; and a generation device configured to generate, by utilizing the respective values of the combination of the at least two kinds of solid angles, an image feature template so as to conduct image feature extraction. 